Electronic device, information providing system, control method, and control program

ABSTRACT

According to one of aspects, an electronic device includes: a camera; a controller configured to determine whether a measured value of a healthcare device is included in an image captured by the camera; and a communication unit configured to transmit information related to the measured value to an information providing device when the measured value is included in the image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a National Stage of PCT international applicationSer. No. PCT/JP2013/072915 filed on Aug. 27, 2013 which designates theUnited States, incorporated herein by reference, and which is based uponand claims the benefit of priority from Japanese Patent Applications No.2012-188808 filed on Aug. 29, 2012, No. 2012-195484 filed on Sep. 5,2012, and No. 2012-197763 filed on Sep. 7, 2012, the entire contents ofwhich are incorporated herein by reference.

FIELD

The present application relates to an electronic device, an informationproviding system, a control method, and a control program.

BACKGROUND

Various measuring instruments are used for healthcare, such as abathroom scale for measuring body weight, a blood pressure monitor formeasuring blood pressure, and a pedometer for counting steps. Some ofthe measuring instruments include those that have a communicationfunction to transmit a measured value to an information processingdevice (see, for example, Patent Literature 1 and Patent Literature 2).The value transmitted to the information processing device is used forvarious analyses.

CITATION LIST Patent Literature

Patent Literature 1: JP-A-2008-033834

Patent Literature 2: JP-A-2005-319216

Technical Problem

When individual measuring instruments are provided with thecommunication function, the configuration thereof is complicated, whichleads to an increase in costs, and makes downsizing difficult. For theforegoing reasons, there is a need for an electronic device, aninformation providing system, a control method, and a control programcapable of transmitting the value measured by a measuring instrument toanother device without complicating the configuration of the measuringinstrument.

SUMMARY

According to one of aspects, an electronic device includes: a camera; acontroller configured to determine whether a measured value of ahealthcare device is included in an image captured by the camera; and acommunication unit configured to transmit information related to themeasured value to an information providing device when the measuredvalue is included in the image.

According to one of aspects, an information providing system includes anelectronic device and an information providing device. The electronicdevice includes: a camera; a first controller configured to determinewhether a measured value of a healthcare device is included in an imagecaptured by the camera; and a communication unit configured to transmitinformation related to the measured value to the information providingdevice when the measured value is included in the image. The informationproviding device includes: a storage configured to store the informationrelated to the measured value; and a second controller configured toperform analysis based on the information related to the measured value.

According to one of aspects, a control method is of an electronic devicewith a camera. The control method includes: determining whether ameasured value of a healthcare device is included in an image capturedby the camera; storing information related to the measured value in astorage; and performing analysis for providing information based on thestored information related to the measured value.

According to one of aspects, a control program causes an electronicdevice with a camera to execute: determining whether a measured value ofa healthcare device is included in an image captured by the camera;storing information related to the measured value in a storage; andperforming analysis for providing information based on the storedinformation related to the measured value.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram of a configuration of an information providingsystem according to one of embodiments.

FIG. 2 is a diagram of an outline of an operation of the informationproviding system.

FIG. 3 is a block diagram of a mobile phone.

FIG. 4 is a diagram of one of examples of a body composition meterincluded in an image.

FIG. 5 is a diagram of one of examples of an activity meter included inan image.

FIG. 6 is a diagram of one of examples of a blood pressure monitorincluded in an image.

FIG. 7 is a block diagram of an information providing device.

FIG. 8 is a diagram of one of examples of measured value information.

FIG. 9 is a flowchart of a processing procedure for reading andtransmitting a measured value.

FIG. 10 is a diagram of one of examples of screen transition.

FIG. 11 is a diagram of a modification of the operation of theinformation providing system.

FIG. 12 is a diagram of another modification of the operation of theinformation providing system.

FIG. 13 is a diagram of another modification of the operation of theinformation providing system.

FIG. 14 is a diagram of another configuration of the informationproviding system.

FIG. 15 is a diagram of one of examples of when an indicator of the bodycomposition meter is captured under a light source.

FIG. 16 is a diagram of one of examples of an image in which a shadow ofthe mobile phone unexpectedly appears.

FIG. 17 is a diagram of another example of the image in which the shadowof the mobile phone unexpectedly appears.

FIG. 18 is a diagram of one of examples of an image captured in anorientation different from that of an indicator.

FIG. 19 is a diagram of one of examples of an image captured in the sameorientation as that of an indicator.

FIG. 20 is a flowchart of one of examples of control for matching anorientation of a captured image and an orientation of an indicator of ahealthcare device included in the image.

FIG. 21 is a diagram of one of examples of a state in which image datais captured by a camera.

FIG. 22 is a diagram of one of examples of a luminance distribution ofthe image data illustrated in FIG. 21.

FIG. 23 is a diagram of one of examples of positions of pixels, in theimage data, having a peak luminance value in the luminance distributionillustrated in FIG. 22.

FIG. 24 is a diagram of one of examples of an area to be cut out as acandidate of an area in which an image of an indicator of a bodycomposition meter appears.

FIG. 25 is a diagram for explaining one of examples of a processingprocedure in which the mobile phone specifies an area, in which an imageof an indicator of a healthcare device appears, from the image datacaptured by the camera and performs photometric processing targeted atthe specified area.

FIG. 26 is a diagram of one of examples of a pattern design forspecifying an area in the image data in which an image of an indicatorof a body composition meter appears.

FIG. 27 is a diagram of one of examples of how the mobile phone itselfunexpectedly appears in an image.

FIG. 28 is a diagram of one of examples of an image in which the mobilephone itself unexpectedly appears.

FIG. 29 is a diagram of one of examples of how a shadow of the mobilephone unexpectedly appears in an image.

FIG. 30 is a diagram of one of examples of an image in which a shadow ofthe mobile phone unexpectedly appears.

FIG. 31 is a diagram of one of examples of a state in which theunexpected appearance does not occur.

FIG. 32 is a diagram of one of examples of an image captured in thestate illustrated in FIG. 31.

FIG. 33 is a diagram of one of examples of an image capture screen.

FIG. 34 is a diagram of one of examples of the image capture screen in astate where the mobile phone faces the healthcare device substantiallydirectly.

FIG. 35 is a diagram of one of examples of the image capture screen inthe state illustrated in FIG. 31.

FIG. 36 is a flowchart of one of examples of control for reducing apossibility for occurrence of the unexpected appearance.

FIG. 37 is a diagram of one of examples in which trapezoidal correctionis performed on the captured image in the state illustrated in FIG. 31.

FIG. 38 is a diagram of another example of the image capture screen.

FIG. 39 is a diagram of one of examples of the image capture screen in astate where the mobile phone faces the healthcare device substantiallydirectly.

FIG. 40 is a diagram of one of examples of the image capture screen inthe state illustrated in FIG. 31.

DESCRIPTION OF EMBODIMENTS

Exemplary embodiments for implementing the present invention will beexplained in detail below with reference to the accompanying drawings.In the following embodiments, a mobile phone will be explained as one ofexamples of the electronic device.

Embodiment 1

A configuration of an information providing system according to one ofembodiments will be explained with reference to FIG. 1. FIG. 1 is adiagram of the configuration of the information providing systemaccording to the embodiment. As illustrated in FIG. 1, the informationproviding system includes a mobile phone 100, a base station 200,information providing devices 300 a and 300 b, and a voice recognizingdevice 400. In the description below, the information providing devices300 a and 300 b may be collectively referred to as an informationproviding device 300 without specifying which is which.

The mobile phone 100 has a function of reading a measured value measuredby a healthcare device and transmitting the read value to theinformation providing device 300. The healthcare device is a measuringinstrument for measuring numerical values related to health. Examples ofthe healthcare device include, but are not limited to, a bathroom scalefor measuring body weight, a body composition meter for measuring bodyfat percentage and the like in addition to the body weight, athermometer for measuring body temperature, a blood pressure monitor formeasuring blood pressure, a pulse meter for measuring pulses, apedometer for counting steps, and an activity meter for measuring anamount of activity due to exercise including walking.

The mobile phone 100 captures the healthcare device by a camera to reada measured value displayed in the indicator provided in the healthcaredevice. By capturing the measured value displayed in the indicator usingthe camera in this way, the mobile phone 100 can read the measured valuemeasured by the healthcare device even if the healthcare device does nothave a communication function. The read measured value is transmitted tothe information providing device 300 through the communication functionprovided in the mobile phone 100. The details of how the mobile phone100 reads the measured value will be explained later.

The base station 200, the information providing device 300, and thevoice recognizing device 400 are communicably connected to each otherthrough a network 500.

The base station 200 establishes a radio communication path with themobile phone 100 located within a communication range, which allows themobile phone 100 to communicate with other device via the radiocommunication path. Hereinafter, when the communication of the mobilephone 100 is to be explained, description of the base station 200 may beomitted for the sake of simplicity of description.

The information providing device 300 stores the measured valuetransmitted from the mobile phone 100, in association with an ID(identifier) of a user of the mobile phone 100 and a date and time(timestamp). The information providing device 300 can store measuredvalues of a plurality of types of healthcare devices in association withrespective identifiers of users. The information providing device 300performs analysis processing based on the measured values accumulated inthis way and transmits information such as advice about health to themobile phone 100. There are a plurality of information providing devices300, and each of them is operated by a different operator.

The voice recognizing device 400 analyzes the content of voiceinformation transmitted from other device by performing voicerecognition processing thereon, and transmits the analyzed content to asource device. For example, when the mobile phone 100 transmits voiceinformation in which user's voice is recorded to the voice recognizingdevice 400, the voice recognizing device 400 analyzes the content of thevoice information through the voice recognition processing and transmitsthe analyzed content as text data to the mobile phone 100. With thissystem, a voice operation of the mobile phone 100 is implemented.

The configuration of the information providing system is not limited tothe example illustrated in FIG. 1. For example, the number of variousdevices included in the information providing system may be larger orsmaller than the number illustrated in FIG. 1. The information providingsystem does not have to include the voice recognizing device 400. Theinformation providing device 300 may be a personal computer in the homeand the base station 200 may be a wireless LAN router in the home.

The outline of an operation of the information providing system will beexplained with reference to FIG. 2. FIG. 2 is a diagram of the outlineof the operation of the information providing system. It is assumed thata healthcare device performs measurement and measured values aredisplayed in the indicator of the healthcare device before the operationillustrated in FIG. 2. In this state, the mobile phone 100 captures animage of the healthcare device using a camera (Step S11). The mobilephone 100 then determines a position and a type of a measured valueincluded in a captured image (Step S12). In the present application,capturing an image is not limited to acquiring an image when a shutteris released (when a release button is pressed), but includes, likeso-called a live view, acquiring continuously videos (images) beingcaptured while displaying them on the display.

After the determination of the position and the type of the measuredvalue in this way, the mobile phone 100 reads the measured value fromthe captured image (Step S13). Reading the measured value included inthe image can be implemented by using known character recognitiontechnology. The mobile phone 100 transmits the read measured value alongwith additional information to the information providing device 300(Step S14). The additional information is necessary information forusing the measured value in the information providing device 300 foranalysis. Examples of the additional information include, but are notlimited to, a value indicating a type of the measured value, a valueindicating a unit of the measured value, etc.

When a plurality of measured values are read from the captured image,the mobile phone 100 may separately transmit the measured values or maycollectively transmit the measured values. The mobile phone 100 maystore the measured values acquired from a plurality of types ofhealthcare devices and collectively transmit the acquired measuredvalues to the information providing device 300 at a certain timing. Inthis case, the mobile phone 100 stores dates and times on which themeasured values are acquired in association with the respective measuredvalues and includes the stored dates and times in the additionalinformation at the time of transmission.

When there are a plurality of information providing devices 300, themobile phone 100 may transmit the measured values and the additionalinformation to a preset information providing device 300, or maydetermine an information providing device 300 as a destination accordingto information such as a maker and a type acquired from the capturedimage.

The information providing device 300 stores the measured valuestransmitted from the mobile phone 100 in association with an ID of auser of the mobile phone 100 and the timestamp (Step S15). The ID of theuser may be acquired from the additional information or may be acquiredbased on a header value of a packet for transmitting the measured valuesfrom the mobile phone 100 to the information providing device 300. Thetimestamp may be acquired from the additional information, or may beacquired based on a value of a timer unit of the information providingdevice 300 of when the measured value is received. When acquired fromthe additional information, the timestamp indicates a date and time onwhich the measured value is read or a date and time on which themeasured value is transmitted, and when acquired based on the value ofthe timer unit of the information providing device 300, the timestampindicates a date and time on which the measured value is received.

Thereafter, the information providing device 300 performs analysisprocessing based on the stored measured value (Step S16). The analysisprocessing may be performed each time when any measured value isreceived, or may be performed each time when a specific type of measuredvalue (for example, body weight) is received. Alternatively, theanalysis processing may be performed at a preset time.

In the analysis processing, analysis about health is performed based onchronological changes or the like of the measured values related to theuser of the mobile phone 100. Because measured values related to aplurality of users are accumulated in the information providing device300, the information providing device 300 may use the informationobtained by statistically processing the measured values related to theusers, in the analysis processing.

The information providing device 300 transmits an analysis result to themobile phone 100 (Step S17). The mobile phone 100 displays the receivedanalysis result (Step S18). The analysis result may be transmitted tothe mobile phone 100 as electronic mail or may be transmitted to themobile phone 100 in any other predetermined format. It may be configuredsuch that the measured value and the additional information aretransmitted as HTTP (HyperText Transfer Protocol) request and theanalysis result is responded in HTML (HyperText Markup Language) format.

In this way, by accumulating the measured values read from thehealthcare device and analyzing them, the information difficult for theuser to notice from individual measured values can be provided to theuser.

The configuration of the mobile phone 100 will be explained withreference to FIG. 3. FIG. 3 is a block diagram of the mobile phone 100.As illustrated in FIG. 3, the mobile phone 100 includes a touch screendisplay 2, a button 3, an illumination sensor 4, a proximity sensor 5, acommunication unit 6, a receiver 7, a microphone 8, a storage 9, acontroller 10, a speaker 11, a camera 12, a camera 13, a connector 14,an acceleration sensor 15, a direction sensor 16, a gyroscope 17, and aGPS (Global Positioning System) receiver 18.

The touch screen display 2 includes a display 2A and a touch screen 2B.The display 2A is provided with a display device such as an LCD (LiquidCrystal Display), an OELD (Organic Electro-Luminescence Display), or anIELD (Inorganic Electro-Luminescence Display). The display 2A displaystext, images, symbols, graphics, and the like.

The touch screen 2B detects a contact of a finger, a pen, a stylus pen,or the like on the touch screen 2B. The touch screen 2B can detectpositions where a plurality of fingers, pens, stylus pens, or the likemake contact with the touch screen 2B. In the following explanation, afinger, a pen, a stylus pen, or the like which comes in contact with thetouch screen 2B may be called “contact object”

The detection method of the touch screen 2B may be any detection methodsuch as a capacitive type detection method, a resistive type detectionmethod, a surface acoustic wave type (or ultrasonic type) detectionmethod, an infrared type detection method, an electro magnetic inductiontype detection method, and a load sensing type detection method. In thedescription herein below, for the sake of simplicity, it is assumed thatthe user uses his/her finger(s) to make contact with the touch screen 2Bin order to operate the mobile phone 100.

The mobile phone 100 determines a type of a gesture based on at leastone of a contact detected by the touch screen 2B, a position where thecontact is detected, a change of the position where the contact isdetected, an interval between detected contacts, and the number ofdetection times of the contact. The gesture is an operation performed onthe touch screen 2B. Examples of the gestures determined by the mobilephone 100 include, but are not limited to, touch, long touch, release,swipe, tap, double tap, long tap, drag, flick, pinch in, and pinch out.In the description herein below, for the sake of simplicity, a case of“the touch screen detects a contact and the mobile phone 100 determinesthat the type of the gesture is X based on the contact” may be describedas “the mobile phone 100 detects X”, “the controller detects X”, or “thetouch screen detects X”. The controller will be explained later.

The button 3 is operated by the user. The controller 10 detects anoperation for the button 3 in cooperation with the button 3. Examples ofthe operations for the button 3 include, but are not limited to, aclick, a double click, a triple click, a push, and a multi-push.

The illumination sensor 4 detects illumination of the ambient light ofthe mobile phone 100. The illumination indicates intensity of light,lightness, or brightness. The illumination sensor 4 is used, forexample, to adjust the brightness of the display 2A. The proximitysensor 5 detects the presence of a nearby object without any physicalcontact. The proximity sensor 5 detects the presence of the object basedon a change of the magnetic field or a change of the return time of thereflected ultrasonic wave, etc. The proximity sensor 5 detects that, forexample, the touch screen display 2 is brought close to someone's face.The illumination sensor 4 and the proximity sensor 5 may be configuredas one sensor. The illumination sensor 4 can be used as a proximitysensor.

The communication unit 6 performs communication via radio waves. Acommunication system supported by the communication unit 6 is wirelesscommunication standard. The wireless communication standard includes,for example, a communication standard of cellular phones such as 2G, 3G,and 4G. The communication standard of cellular phones includes, forexample, LTE (Long Term Evolution), W-CDMA (Wideband Code DivisionMultiple Access), CDMA 2000, PDC (Personal Digital Cellular), GSM(registered trademark) (Global System for Mobile Communications), andPHS (Personal Handy-phone System). The wireless communication standardfurther includes, for example, WiMAX (Worldwide Interoperability forMicrowave Access), IEEE 802.11, Bluetooth (registered trademark), IrDA(Infrared Data Association), and NFC (Near Field Communication). Thecommunication unit 6 may support one or more communication standards.

The receiver 7 and the speaker 11 are sound output units. The receiver 7and the speaker 11 output a sound signal transmitted from the controller10 as sound. The receiver 7 is used, for example, to output voice of theother party on the phone call. The speaker 11 is used, for example, tooutput a ring tone and music. One of the receiver 7 and the speaker 11may double as the other function. The microphone 8 is a sound inputunit. The microphone 8 converts voice of the user or the like to a soundsignal and transmits the converted signal to the controller 10.

The storage 9 stores therein programs and data. The storage 9 is usedalso as a work area that temporarily stores a processing result of thecontroller 10. The storage 9 may include any non-transitory storagemedium such as a semiconductor storage medium and a magnetic storagemedium. The storage 9 may include a plurality of types of storagemediums. The storage 9 may include a combination of a portable storagemedium such as a memory card, an optical disc, or a magneto-optical discwith a reader of the storage medium. The storage 9 may include a storagedevice used as a temporary storage area such as Random Access Memory(RAM).

Programs stored in the storage 9 include applications executed in theforeground or the background and a control program for assistingoperations of the applications. The application causes the controller10, for example, to display a screen on the display 2A and performprocessing according to a gesture detected through the touch screen 2B.The control program is, for example, an OS. The applications and thecontrol program may be installed in the storage 9 through wirelesscommunication by the communication unit 6 or through a non-transitorystorage medium.

The storage 9 stores, for example, a control program 9A, a mailapplication 9B, a browser application 9C, a healthcare application 9D,used device information 9X, determination information 9Y, and settingdata 9Z. The control program 9A provides functions related to variouscontrols for operating the mobile phone 100. The control program 9Aactivates or terminates an application according to a detectedoperation, for example.

The mail application 9B provides an e-mail function for composition,transmission, reception, and display of e-mail, and the like. Thebrowser application 9C provides a WEB browsing function for display ofWEB pages.

The healthcare application 9D provides a function of reading a measuredvalue displayed in an indicator of a healthcare device and transmittingthe read value to the information providing device 300, and a functionof displaying an analysis result transmitted from the informationproviding device 300 on the display 2A.

The used device information 9X includes information indicating featureson the appearance of a healthcare device used by the user of the mobilephone 100 (hereinafter, it may be called “used device”) and informationindicating a position and a type of the measured value displayed by thehealthcare device. The information indicating the features on theappearance of the healthcare device includes at least one of, forexample, a shape (shape of the whole or part of the healthcare device),a color, a typeface of a measured value, a layout of the measured value,a character string or a symbol near the measured value, and othercharacter or symbol on the surface of the healthcare device and itsposition. When a plurality of measured values are simultaneouslydisplayed in the indicator of the healthcare device, informationindicating the position and the type of the measured value includesinformation corresponding to each measured value. When the measuredvalues displayed in the indicator of the healthcare device are changed,information indicating the position and the type of the measured valueincludes information related to the order of displaying the measuredvalues.

As explained later, information is added to the used device information9X each time the mobile phone 100 reads a measured value from a newhealthcare device. The used device information 9X may initially have noinformation, or may store information corresponding to previouslyselected healthcare devices.

The determination information 9Y includes information for determiningwhich measured value is present in which part of a captured image.Specifically, the determination information 9Y stores the informationrelated to the features on the appearance and the information indicatingthe position and the type of a measured value displayed by a healthcaredevice in association with each other, for each healthcare deviceexisting on the market. The information indicating the features on theappearance of a healthcare device includes at least one of, for example,a shape (shape of the whole or part of the healthcare device), a color,a typeface of a measured value, a layout of a measured value, acharacter string or a symbol near the measured value, and othercharacter or symbol on the surface of the healthcare device and itsposition. When a plurality of measured values are simultaneouslydisplayed in the indicator of the healthcare device, informationindicating the position and the type of the measured value includesinformation corresponding to each measured value. When the measuredvalues displayed in the indicator of the healthcare device are changed,information indicating the position and the type of the measured valueincludes information related to the order of displaying the measuredvalues.

The determination information 9Y may combine a plurality of models whosefeatures on the appearance are substantially the same as each other andin which the position and the type of the displayed measured value isthe same as each other, to store the information as one healthcaredevice. The determination information 9Y does not have to cover theinformation related to all the healthcare devices existing on themarket.

The determination information 9Y also includes abstracted information soas to read a measured value even from an unknown healthcare device.Specifically, the determination information 9Y stores informationrelated to a character or a symbol likely to be present near themeasured value, for each type of the measured value. In general, atleast one of a name and a unit of a measured value is present near themeasured value. By storing such a name or a unit associated with thetype of the measured value, it becomes possible to detect a measuredvalue near the name or the unit by using it as a clue and determine thetype of the measured value. In this case, to facilitate detection of ameasured value, a relative position of the measured value correspondingto the name or the unit may be stored along with the name and the unit.For example, when a unit is stored, description indicating the presenceof a measured value on the left side of the unit may further be stored.

The determination information 9Y further stores information, asabstracted information, related to a range of measured values for eachtype of measured value. For example, when the measured value is a bodytemperature, a range from 35° C. to 40° C. is stored as its range. Forexample, when the measured value is systolic blood pressure, a rangefrom 100 mmHg to 200 mmHg is stored as its range. When a numerical valueis detected in the image, by checking the numerical value against theranges, it is possible to determine whether the numerical value is ameasured value and which type of measured value it is.

The determination information 9Y further stores information, asabstracted information, related to a layout of measured values. When aplurality of measured values are simultaneously displayed, the layout ofmeasured values may have a regularity in each maker or each type ofhealthcare device. For example, when it is a pulse meter, systolic bloodpressure, diastolic blood pressure, the number of pulses are, in manycases, vertically displayed in the order from the top. When a pluralityof numerical values are detected in the image, by checking the layout ofthe numerical values against the regularity, it is possible to determinewhether the numerical values are measured values and which type of themeasured value each of them is.

The determination information 9Y also stores information used todetermine a maker and a type of a healthcare device. Specifically, thedetermination information 9Y stores information related to a typeface ofa displayed measured value for each maker. Furthermore, thedetermination information 9Y stores information related to a name or asymbol of a company to be printed on, to be stamped on, or to beattached to the surface of the healthcare device for each maker.Moreover, the determination information 9Y stores information related toa model number to be printed on, to be stamped on, or to be attached tothe surface of the healthcare device for each maker and for each type ofthe healthcare device. The determination information 9Y storesinformation used to determine a maker and a type of the healthcaredevice associated with the information related to the features on theappearance of the healthcare device. The abstracted information includedin the determination information 9Y may be stored for each maker or eachtype of the healthcare device.

The setting data 9Z includes information related to various settings forthe operation of the mobile phone. 100.

The controller 10 is a processing unit. Examples of the processing unitinclude, but are not limited to, a CPU (Central Processing Unit), an SoC(System-on-a-chip), an MCU (Micro Control Unit), and an FPGA(Field-Programmable Gate Array). The controller 10 integrally controlsthe operations of the mobile phone 100 to implement the variousfunctions.

Specifically, the controller 10 executes instructions included in theprogram stored in the storage 9 while referring to the data stored inthe storage 9 as necessary. The controller 10 controls a function unitaccording to the data and the instructions to thereby implement thevarious functions. Examples of the function unit include, but are notlimited to, the display 2A, the communication unit 6, the receiver 7,and the speaker 11. The controller 10 can change the control accordingto the detection result of a detector. Examples of the detector include,but are not limited to, the touch screen 2B, the button 3, theillumination sensor 4, the proximity sensor 5, the microphone 8, thecamera 12, the camera 13, the acceleration sensor 15, the directionsensor 16, the gyroscope 17, and the GPS receiver 18.

The controller 10 executes, for example, the healthcare application 9Dto thereby implement a function of reading a measured value displayed inan indicator of a healthcare device and transmitting the read measuredvalue to the information providing device 300 and a function ofdisplaying an analysis result transmitted from the information providingdevice 300 on the display 2A.

The camera 12 is an in-camera for capturing an object facing the frontface. The camera 13 is an out-camera for capturing an object facing theback face. The camera 13 is also used to capture the healthcare device.

The connector 14 is a terminal to which another device is connected. Theconnector 14 may be a general-purpose terminal such as a USB (UniversalSerial Bus), a HDMI (registered trademark) (High-Definition MultimediaInterface), Light Peak (Thunderbolt (registered trademark)), and anearphone/microphone connector. The connector 14 may be a dedicatedterminal such as a Dock connector. Examples of the device connected tothe connector 14 include, but are not limited to, an external storage, aspeaker, and a communication device.

The acceleration sensor 15 detects a direction and a magnitude ofacceleration applied to the mobile phone 100. The direction sensor 16detects a direction of geomagnetism. The gyroscope 17 detects an angleand an angular velocity of the mobile phone 100. The detection resultsof the acceleration sensor 15, the direction sensor 16, and thegyroscope 17 are used in combination with each other in order to detecta position of the mobile phone 100 and a change of its attitude. The GPSreceiver 18 detects a position of the mobile phone 100.

Part or all of the programs and the data stored in the storage 9 in FIG.3 may be downloaded from any other device through wireless communicationby the communication unit 6. Part or all of the programs and the datastored in the storage 9 in FIG. 3 may be stored in a non-transitorystorage medium that can be read by a reader included in the storage 9.Part or all of the programs and the data stored in the storage 9 in FIG.3 may be stored in a non-transitory storage medium that can be read by areader connected to the connector 14. Examples of the non-transitorystorage mediums include, but are not limited to, an optical disc such asCD (registered trademark), DVD (registered trademark), and Blu-ray(registered trademark), a magneto-optical disc, a magnetic storagemedium, a memory card, and a solid-state storage medium.

The configuration of the mobile phone 100 illustrated in FIG. 3 isexemplarily illustrated, and therefore it can be modified as requiredwithin a scope that does not depart from the gist of the presentinvention. For example, in the example illustrated in FIG. 3, the mobilephone 100 is provided with two cameras; however, the mobile phone 100may be provided with only the camera 13. In the example of FIG. 3, themobile phone 100 is provided with four types of sensors in order todetect its position and attitude; however, the mobile phone 100 does nothave to be provided with some of the sensors. Alternatively, the mobilephone 100 may be provided with any other type of sensor for detecting atleast one of the position and the attitude.

Examples of how the mobile phone 100 reads a measured value will beexplained with reference to FIG. 4 to FIG. 6. FIG. 4 is a diagram of oneof examples of a body composition meter 40 included in an image capturedby the camera 13. The body composition meter 40 includes an indicator41, a maker name 44 printed on the surface, and electrodes 45 formeasuring body fat percentage. A measured value 42 a indicating bodyweight and a measured value 43 a indicating body fat percentage aredisplayed in the indicator 41. Furthermore, a unit 42 b indicating thata unit of the body weight is Kg is displayed adjacently on the rightside of the measured value 42 a, and a symbol 43 b indicating that thebody fat percentage is expressed in percentage is displayed adjacentlyon the right side of the measured value 43 a.

When the body composition meter 40 as described above is included in theimage, the mobile phone 100 detects the measured value 42 a and themeasured value 43 a which are numerical values, the unit 42 b adjacentto the measured value 42 a, and the symbol 43 b adjacent to the measuredvalue 43 a. Moreover, the mobile phone 100 acquires information relatedto the features on the appearance of the body composition meter 40including the measured value 42 a and the measured value 43 a. Forexample, a shape 40 a of the body composition meter 40, a color of thebody composition meter 40, a position of the indicator 41, a characterstyle and layout of the information displayed in the indicator 41,characters forming the maker name 44, a character style and position ofthe maker name 44, positions of the electrode 45 and the electrode 45are acquired as the information related to the features on theappearance.

The mobile phone 100 checks the acquired pieces of information againstat least one of the used device information 9X and the determinationinformation 9Y, to thereby determine that the measured value 42 aindicates the body weight and the measured value 43 a indicates the bodyfat percentage. The mobile phone 100 then acquires the body weight andthe body fat percentage as numerical value data through the characterrecognition processing.

FIG. 5 is a diagram of one of examples of an activity meter 50 includedin an image captured by the camera 13. The activity meter 50 includes anindicator 51, a model number 54 printed on the surface, a button 55, anda button 56. A measured value 52 a indicating steps and a measured value53 a indicating consumed calorie are displayed in the indicator 51.Furthermore, a unit 52 b indicating that a unit of the steps is step(STEP) is displayed adjacently on the right side of the measured value52 a, and a unit 53 b indicating that a unit of the consumed calorie isKcal is displayed adjacently on the right side of the measured value 53a.

When the activity meter 50 as described above is included in the image,the mobile phone 100 detects the measured value 52 a and the measuredvalue 53 a which are numerical values, the unit 52 b adjacent to themeasured value 52 a, and the unit 53 b adjacent to the measured value 53a. Moreover, the mobile phone 100 acquires information related to thefeatures on the appearance of the activity meter 50 including themeasured value 52 a and the measured value 53 a. For example, a shape 50a of the activity meter 50, a color of the activity meter 50, a positionof the indicator 51, a character style and layout of the informationdisplayed in the indicator 51, characters forming the model number 54, acharacter style and position of the model number 54, and shapes andpositions of the buttons 55 and 56 are acquired as the informationrelated to the features on the appearance.

The mobile phone 100 checks the acquired pieces of information againstat least one of the used device information 9X and the determinationinformation 9Y, to thereby determine that the measured value 52 aindicates steps and the measured value 53 a indicates consumed calorie.The mobile phone 100 then acquires the steps and the consumed calorie asnumerical value data through the character recognition processing.Timing at which the mobile phone 100 determines that the measured value52 a indicates the steps and the measured value 53 a indicates theconsumed calorie may be after the character recognition processing isperformed.

FIG. 6 is a diagram of one of examples of a blood pressure monitor 60included in an image captured by the camera 13. The blood pressuremonitor 60 includes an indicator 61 and a button 65. A measured value 62a indicating systolic blood pressure, a measured value 63 a indicatingdiastolic blood pressure, and a measured value 64 a indicating pulsesare displayed in the indicator 61. Furthermore, a name 62 b representingthat the measured value 62 a indicates the systolic blood pressure isprinted adjacently on the left side of the measured value 62 a, a name63 b representing that the measured value 63 a indicates the diastolicblood pressure is printed adjacently on the left side of the measuredvalue 63 a, and a name 64 b representing that the measured value 64 aindicates the pulses is printed adjacently on the left side of themeasured value 64 a.

When the blood pressure monitor 60 as described above is included in theimage, the mobile phone 100 detects the measured value 62 a, themeasured value 63 a, and the measured value 64 a which are numericalvalues, the name 62 b adjacent to the measured value 62 a, the name 63 badjacent to the measured value 63 a, and the name 64 b adjacent to themeasured value 64 a. Moreover, the mobile phone 100 acquires informationrelated to the features on the appearance of the blood pressure monitor60 including the measured value 62 a, the measured value 63 a, and themeasured value 64 a. For example, a shape 60 a of the blood pressuremonitor 60, a color of the blood pressure monitor 60, a position of theindicator 61, a character style and layout of the information displayedin the indicator 61, characters forming each of the names 62 b to 64 b,a character style and position of each of the names 62 b to 64 b, and ashape and position of the button 65 are acquired as the informationrelated to the features on the appearance.

The mobile phone 100 checks the acquired pieces of information againstat least one of the used device information 9X and the determinationinformation 9Y, to thereby determine that the measured value 62 aindicates systolic blood pressure, the measured value 63 a indicatesdiastolic blood pressure, and the measured value 64 a indicates pulses.The mobile phone 100 then acquires the systolic blood pressure, thediastolic blood pressure, and the pulses as numerical value data throughthe character recognition processing.

A configuration of the information providing device 300 will beexplained with reference to FIG. 7 and FIG. 8. FIG. 7 is a block diagramof the information providing device 300. FIG. 8 is a diagram of one ofexamples of measured value information. As illustrated in FIG. 7, theinformation providing device 300 includes a communication unit 301, acontroller 302, and a storage 303. The communication unit 301 allows theinformation providing device 300 to communicate with other devices basedon a predetermined protocol.

The controller 302 is a processing unit such as a CPU (CentralProcessing Unit). The controller 302 integrally controls the operationsof the information providing device 300 to implement the variousfunctions. Specifically, the controller 302 executes instructionsincluded in the program stored in the storage 303 while referring to thedata stored in the storage 303 as necessary. The controller 10 thenexecutes various information processing according to the data and theinstructions.

The controller 302 performs, for example, an analysis program 304 tothereby perform analysis processing based on the stored measured values.

The storage 303 stores therein programs and data. The storage 303 isused also as a work area that temporarily stores a processing result ofthe controller 302. The storage 303 may include any non-transitorystorage medium such as a semiconductor storage medium and a magneticstorage medium. The storage 303 may include a plurality of types ofstorage mediums. The storage 303 may include a combination of a portablestorage medium such as a memory card, an optical disc, or amagneto-optical disc with a reader of the storage medium. The storage303 may include a storage device used as a temporary storage area suchas RAM (Random Access Memory).

The storage 303 stores, for example, the analysis program 304 andmeasured value information 306. The analysis program 304 provides afunction for performing the analysis processing based on the storedmeasured values.

The measured values transmitted from the mobile phone 100 areaccumulated in the measured value information 306. FIG. 8 depicts one ofexamples of the measured value information 306. As illustrated in FIG.8, the measured value information 306 includes items such as user ID,timestamp, and type, and a plurality of items in which a measured valueand a name of its type are stored in association with each other. Storedin the item of the user ID is an identifier to identify the user of themobile phone 100. Stored in the item of the timestamp is a date and timewhen the measured value is read, a date and time when the measured valueis transmitted, or a date and time when the measured value is received.Stored in the item of the type is a value indicating the type of thehealthcare device from which the measured value is read.

As the example illustrated in FIG. 8, the measured value information 306is structured so that a plurality of types of measured values can bestored in chronological order. Therefore, the information providingdevice 300 can perform a high-level analysis related to health based onchronological changes in a plurality of types of measured values.

A processing procedure of the mobile phone 100 that reads and transmitsa measured value will be explained with reference to FIG. 9 and FIG. 10.FIG. 9 is a flowchart of a processing procedure for reading andtransmitting a measured value. FIG. 10 is a diagram of one of examplesof screen transition of when the measured value is read and transmitted.The operations illustrated in FIG. 9 and FIG. 10 are implemented by thecontroller 10 executing the healthcare application 9D.

To perform the procedure illustrated in FIG. 9, first of all, thecontroller 10 activates the healthcare application 9D. The activation ofthe healthcare application 9D is performed in response to, for example,detection of a user operation. The operation to activate the healthcareapplication 9D may be an operation performed on a menu or an operationperformed on an icon similarly to the operation to activate otherapplications. Alternatively, the operation to activate the healthcareapplication 9D may be an operation on a lock screen.

The lock screen is a screen on which an operation for shifting themobile phone 100 from a standby state to a normal state. At Step S21illustrated in FIG. 10, a lock screen 31 is displayed on the display 2A.An unlock icon 31 a, a camera icon 31 b, and a measured value readingicon 31 c are laid out on the lock screen 31.

When an operation of moving the unlock icon 31 a in the direction of thecamera icon 31 b with a flick or a drag is detected through the touchscreen 2B, the controller 10 activates an application for photography.When an operation of moving the unlock icon 31 a in the direction of themeasured value reading icon 31 c with a flick or a drag is detectedthrough the touch screen 2B, the controller 10 activates the healthcareapplication 9D. When an operation of moving the unlock icon 31 a in anyother direction with a flick or a drag is detected through the touchscreen 2B, the controller 10 displays a home screen or a screendisplayed before the shift to the standby state on the display 2A.

By configuring so that the healthcare application 9D can be activated byan operation performed on the lock screen in this way, the user canrapidly start read of the measured value from the healthcare device.

When the healthcare application 9D is activated, the controller 10 firstactivates the camera 13 (Step S101), as illustrated in FIG. 9. Then, thecontroller 10 captures an image using the camera 13 (Step S102), anddisplays the captured image on the display 2A (Step S103).

At this stage, as illustrated at Step S22 in FIG. 10, an image capturescreen 32 for capturing an image is displayed on the display 2A. Whenthe user standing on the body composition meter directs the camera 13toward the body composition meter, the body composition meter displayingthe body weight and the body fat percentage in the indicator iscaptured, as illustrated at Step S22.

Subsequently, the controller 10 detects a numeral in the image capturedby the camera 13 (Step S104). When the numeral cannot be detected in theimage, that is, when the healthcare device that displays the measuredvalue is not captured (No at Step S105), the controller 10 re-executesStep S102 and subsequent steps.

When the numeral in the image captured by the camera 13 can be detected(Yes at Step S105), the controller 10 detects characters near thenumeral (Step S106). Moreover, the controller 10 acquires the shape andthe color of the device including the numeral (Step S107), and detectsother characters and symbols included in the device (Step S108). Thecontroller 10 then acquires a layout of the detected numeral,characters, and symbols (Step S109). The order of Steps S106 to S108 isnot limited thereto.

After the information related to the features on the appearance of thehealthcare device is acquired in this way, the controller 10 checks theacquired information against the used device information 9X (Step S110).When there is a healthcare device whose matching degree of theinformation related to the features on the appearance is higher than apredetermined value, that is, when the used device in the image isdetected (Yes at Step S111), the controller 10 executes Step S115 andsubsequent steps.

When the used device in the image is not detected (No at Step S111), thecontroller 10 checks the acquired information related to the features onthe appearance of the healthcare device against the determinationinformation 9Y in order to detect a healthcare device other than theused device (Step S112). When the healthcare device in the image cannotbe specified through checking (No at Step S113), the controller 10re-executes Step S102 and subsequent steps.

When the healthcare device in the image can be specified throughchecking (Yes at Step S113), the controller 10 adds the information ofthe specified healthcare device to the used device information 9X (StepS114). Specifically, the information related to the features on theappearance acquired at Step S104 to Step S109 is added to the useddevice information 9X in association with the information indicating theposition and type of measured value of the data in the determinationinformation 9Y corresponding to the healthcare device specified at StepS113. In this way, by storing the information for the newly recognizedhealthcare device in the used device information 9X, the processingrelated to the healthcare device is speeded up at the next andsubsequent times. Thereafter, the controller 10 executes Step S115 andsubsequent steps.

At Step S115, the controller 10 displays a detection-result displayscreen on the display 2A (Step S115). The controller 10 then displaysthe maker name and the type of the healthcare device included in thecaptured image on the detection-result display screen (Step S116).

Subsequently, the controller 10 captures an image using the camera 13(Step S117), and displays the captured image on the detection-resultdisplay screen (Step S118). The controller 10 then detects the measuredvalues in the image (Step S119), and displays the detected measuredvalues on the detection-result display screen (Step S120).

At this stage, for example, a detection-result display screen 33 asillustrated at Step S23 in FIG. 10 is displayed on the display 2A. Thedetection-result display screen 33 includes an image display area 33 a,a send button 33 b, and a detection-result display area 33 c. The imagedisplay area 33 a is an area in which the captured image is displayed.The controller 10 enlarges an area, of the captured image, where themeasured values are displayed, and displays the enlarged area in theimage display area 33 a. Therefore, the user can easily grasp themeasured values even in such a situation that the indicator of thehealthcare device cannot be seen well. The mobile phone 100 may beconfigured such that the user can correct the detection result using thetouch screen 2B or the button 3 at Step S120.

The send button 35 is a button for instruction of executing transmissionof the measured values and the additional information to the informationproviding device 300. The detection-result display area 33 c is an areain which the information obtained by analyzing the captured image isdisplayed.

Subsequently, the controller 10 determines whether transmission of themeasured values and the additional information has been instructed (StepS121). When transmission has not been instructed (No at Step S121), thecontroller 10 re-executes Step S117 and subsequent steps. In this way,the controller 10 repeats acquisition of an image, display of the image,detection of measured values, and display of the detected values untilthe transmission is instructed. Therefore, when a measured valueincluded in the image is changed, the mobile phone 100 can reflect thechange in the detection result.

For example, in the case of the body composition meter, it may take sometime until the measured values are stabilized. Even in this case, thecontroller 10 repeatedly performs the detection of measured values, andcan therefore reflect the latest measured value in the detection result.Moreover, in the case of the body composition meter, the type of themeasured value displayed in the indicator may be changed according to auser's operation or according to passage of time. Even in this case, thecontroller 10 repeatedly performs detection of measured values, and cantherefore reflect the newly displayed measured value in the detectionresult as illustrated at Step S24 in FIG. 10. To determine the type ofthe newly displayed measured value, the controller 10 may performdetection of a character or symbol near the measured value and check ofthe detected character or symbol against the determination information9Y, at Step S 119.

Furthermore, the controller 10 repeatedly performs processing ofenlarging the area, of the captured image, where the measured values aredisplayed and displaying the enlarged area in the image display area 33a, and therefore the same portion in the image can be continuouslydisplayed even if the image is blurred due to camera shake. Thus, themobile phone 100 can enhance the visibility of the displayed image.

Moreover, because the analysis of the image is repeatedly performed, theuser can also perform the operation of causing the mobile phone 100 toread the model number of the healthcare device to fix the type of thehealthcare device and the position and type of the measured valuedisplayed in the indicator of the healthcare device, and then causingthe mobile phone 100 to read the measured value.

When the transmission has been instructed (Yes at Step S121), thecontroller 10 instructs the communication unit 6 to transmit thedetected measured value and the additional information to theinformation providing device 300 (Step S122). The transmission isinstructed by, for example, the user, who checks the measured valuedisplayed on the detection-result display screen 33, tapping the sendbutton 33 b using a finger F1 as illustrated at Step S24 in FIG. 10.

Thereafter, when the analysis result is transmitted from the informationproviding device 300, the controller 10 displays the analysis result onthe display 2A. For example, the controller 10 displays, for example, ananalysis result screen 34 illustrated at Step S25 in FIG. 10 on thedisplay 2A. The analysis result screen 34 includes a history displayarea 34 a for displaying chronological changes in a plurality of typesof measured values, and an advice display area 34 b for displaying anadvice obtained by analysis processing.

The embodiments disclosed in the present application can be modifiedwithin a range not departing from the gist and the scope of theinvention. Moreover, the embodiments and the modifications thereofdisclosed in the present application can be appropriately combined witheach other. For example, the embodiment may be modified as follows.

For example, each of the programs illustrated in FIG. 3 and FIG. 7 maybe divided into a plurality of modules or may be combined with anotherprogram.

The arrangement of the functions of the mobile phone 100 and thefunction of the information providing device 300 may be appropriatelychanged. For example, the mobile phone 100 may perform the functions ofthe information providing device 300. That is, the mobile phone 100 mayaccumulate and analyze the measured values in addition to reading themeasured value.

Alternatively, the information providing device 300 may perform part ofthe functions of the mobile phone 100. When the information providingdevice 300 performs part of the functions of the mobile phone 100, theprocessing of determining the position and type of the measured valueincluded in the image may be performed by the information providingdevice 300. In this case, as illustrated in FIG. 11, it may beconfigured such that the mobile phone 100 transmits the image to theinformation providing device 300, and the information providing device300 performs both the checking of the image against the used deviceinformation 9X and the checking thereof against the determinationinformation 9Y. FIG. 11 is a diagram of a modification of the operationof the information providing system. Alternatively, as illustrated inFIG. 12, it may be configured such that the mobile phone 100 checks theimage against the used device information 9X and transmits the image tothe information providing device 300 when the position and the like ofthe measured value cannot be determined, and the information providingdevice 300 checks the image against the determination information 9Y.FIG. 12 is a diagram of another modification of the operation of theinformation providing system. In the example illustrated in FIG. 12, theinformation providing device 300 transmits a determination result andthe used device information 9X corresponding to the determination resultto the mobile phone 100; however, it may be configured to transmit onlythe determination result. In the configuration illustrated in FIG. 11,the load on the mobile phone 100 is reduced and the support of a newmodel of the healthcare device is facilitated. In the configurationillustrated in FIG. 12, it is possible to reduce the time required fordetermination of the position and the like of the measured value whilefacilitating the support of a new model of the healthcare device.

When part of the functions of the mobile phone 100 is performed by theinformation providing device 300, it may be configured such that themobile phone 100 transmits the image and the additional information tothe information providing device 300, and the information providingdevice 300 reads the measured value in addition to determining theposition and the like of the measured value, as illustrated in FIG. 13.FIG. 13 is a diagram of another modification of the operation of theinformation providing system. In the configuration illustrated in FIG.13, the load on the mobile phone 100 can be further reduced.

Alternatively, part of the functions of the mobile phone 100 may beperformed by a device other than the information providing device 300.For example, as illustrated in FIG. 14, an instrument determining device600 for determining a position and the like of a measured value may beadded to the information providing system. FIG. 14 is a diagram ofanother configuration of the information providing system.

The embodiment has explained the examples in which the mobile phone 100reads the measured value displayed in the indicator of the healthcaredevice; however, the way to read the measured value is not limitedthereto. For example, the mobile phone 100 may read a measured valuefrom a positional relation between a pointer and a scale provided in ananalog healthcare device.

When reading a measured value of a measuring instrument such as abathroom scale or a body composition meter for measuring body weight,the user is thought to step on the measuring instrument while holdingthe mobile phone 100. Therefore, to remove the influence which theweight of the mobile phone 100 exerts on the measured value, the mobilephone 100 may be configured such that the weight of the mobile phone 100is stored therein in advance and the weight of the mobile phone 100 isremoved from the measured value indicating the body weight when thedevice displaying the measured value is a measuring instrument formeasuring body weight.

The healthcare device may be configured such that the measured value isdisplayed not only as a numerical value but also as a code that themobile phone 100 can easily read. In this case, the type, the unit, andthe like of the measured value may be encoded, as well as the measuredvalue. The encoding can be implemented by using, for example, aone-dimensional barcode or a two-dimensional barcode. Alternatively,when the indicator of the healthcare device uses a 7-segment display,the encoding may be implemented by assigning a flashing pattern ofelements forming each character to a code.

When user's personal information such as age, height, and gender isregistered in the healthcare device, the healthcare device may beconfigured in such a manner as to display not only the measured valuebut also the personal information in the indicator. By such aconfiguration, the mobile phone 100 acquires personal information notregistered in the mobile phone 100 from the healthcare device andtransmits the acquired personal information to the information providingdevice 300, so that the information can be used for analysis. Thepersonal information may be displayed as text in the indicator of thehealthcare device or may be displayed by being encoded in the abovemanner. The personal information may be stored in the informationproviding device 300 in advance.

The mobile phone 100 may perform control so that an orientation of animage obtained by capturing the indicator is changed according to anaspect ratio of the indicator of the healthcare device. A relationshipbetween an aspect ratio of the indicator and an aspect ratio of theimage will be explained using a case of capturing the indicator 41 ofthe body composition meter 40 illustrated in FIG. 4 as one of examples.In the following explanation, a match between the orientation of theindicator of the healthcare device and the orientation of the capturedimage means a match between a long-side direction of the indicator and along-side direction of the image (a match between a short-side directionof the indicator and a short-side direction of the image). The matchbetween the orientation of the indicator of the healthcare device andthe orientation of the captured image may be meant that an orientationof a measured value displayed in the indicator and of a character orsymbol etc. near the measured value matches the short-side direction ofthe image.

FIG. 15 is a diagram of one of examples of when the indicator 41 of thebody composition meter 40 is captured under a light source. In theexample illustrated in FIG. 15, a light source L1 is provided on aceiling and the body composition meter 40 is placed on a floor. Themobile phone 100 is held by the user in between the light source L1 andthe body composition meter 40 in order to capture the indicator 41 ofthe body composition meter 40 using the camera 13. In this case, ashadow of the mobile phone 100 occurs over the body composition meter 40due to the light from the light source L1, and the shadow is likely tounexpectedly appear in the image captured by the camera 13. In thedescription herein below, for the sake of simplicity, a user's body(especially, hand) by which the mobile phone 100 is held is notconsidered.

FIG. 16 is a diagram of one of examples of an image in which the shadowof the mobile phone 100 unexpectedly appears. A picture image P1illustrated in FIG. 16 is captured in an orientation different from thatof the indicator 41. That is, the picture image P1 illustrated in FIG.16 is captured in an orientation in which a long-side direction of thepicture image P1 does not match the long-side direction of the indicator41 (in an orientation in which a short-side direction of the pictureimage P1 does not match the short-side direction of the indicator 41).Therefore, an orientation of an image P1 a of the indicator 41 in thepicture image P1 does not match the orientation of the picture image P1.

A shadow P1 b of the mobile phone 100 unexpectedly appears in thepicture image P1. The mobile phone 100 is configured such that the touchscreen display 2 thereof is provided over a substantially entire surfaceof the face opposite to the camera 13 and an image being captured by thecamera 13 can be displayed on the substantially entire surface ofitself. Therefore, the orientation of the image captured by the camera13 matches the orientation of the mobile phone 100, and thus theorientation of the shadow P1 b of the mobile phone 100 in the pictureimage P1 is always the same as the orientation of the picture image P1.

In this way, the picture image P1 includes the image P1 a of theindicator 41 in the orientation different from that of the picture imageP1 and the shadow P1 b in the same orientation as that of the pictureimage P1. The images in the different orientations in the above mannerare likely to have a relationship in which only parts of them overlapeach other as the example illustrated in FIG. 16. When the shadow P1 boverlaps part of the image P1 a in this manner, the luminance and thelike inside the image P1 a partially change, and the partial change mayadversely affect the reading of the measured value in the image P1 a.Particularly, when a reflective display panel without a backlight isused for the indicator of the healthcare device, an amount of change inthe luminance of the indicator is originally small, and therefore themeasured value may become difficult to be read due to a luminancedifference produced by the shadow.

FIG. 17 is a diagram of another example of an image in which the shadowof the mobile phone 100 unexpectedly appears. A picture image P2illustrated in FIG. 17 is captured in the same orientation as that ofthe indicator 41. That is, the picture image P2 illustrated in FIG. 17is captured in an orientation in which a long-side direction of thepicture image P2 matches the long-side direction of the indicator 41 (inan orientation in which a short-side direction of the picture image P2matches the short-side direction of the indicator 41). Therefore, anorientation of an image P2 a of the indicator 41 in the picture image P2matches the orientation of the picture image P2.

A shadow P2 b of the mobile phone 100 unexpectedly appears in thepicture image P1. As explained above, the orientation of the imagecaptured by the camera 13 matches the orientation of the mobile phone100. Therefore, the orientation of the shadow P2 b of the mobile phone100 in the picture image P2 matches the orientation of the picture imageP2.

In this way, the picture image P2 includes the image P2 a of theindicator 41 in the same orientation as that of the picture image P2 andthe shadow P2 b in the same orientation as that of the picture image P2.The images in the same orientations in the above manner are likely tohave a relationship in which one is included in the other, as theexample illustrated in FIG. 17. Moreover, in consideration of a generalsize of the indicator of the healthcare device, a general size of themobile phone, a relative positions between the light source, the mobilephone, and the healthcare device in a room as generally assumed, and thelike, it is considered that the shadow P2 b is, in many cases, largerthan the image P2 a of the indicator 41. Therefore, when the pictureimage P2 is captured in the same orientation as that of the indicator41, a relationship in which the image P2 a of the indicator 41 isincluded in the shadow P2 b is likely to be made, as the exampleillustrated in FIG. 17. In such a relationship as above, a difference inthe luminance or the like due to the shadow P2 b is not produced insidethe image P2 a of the indicator 41, and therefore the influence of theshadow P2 b exerted on the reading of the measured value is small evenif the shadow P2 b unexpectedly appears in the image P2 a.

A portion in which photometric processing is performed to adjustexposure (automatic exposure, automatic sensitivity adjustment) mayaffect the reading of the measured value as well as the change of theluminance or the like due to the shadow. When the orientation of theindicator of the healthcare device is different from the orientation ofa captured image, the entire indicator can be captured even if theposition of the indicator in the picture image is largely displaced fromthe center of the picture image. FIG. 18 is a diagram of one of examplesof an image captured in an orientation different from that of theindicator. In a picture image P3 illustrated in FIG. 18, an image P3 aof the indicator 41 includes the entire indicator 41 but is largelydisplaced from the center of the picture image P3. Accordingly, when themobile phone 100 performs photometric processing based on a central areaP3 b as a default operation, exposure adjustment is performed based onbrightness in an area not including the image P3 a of the indicator 41,and therefore it may become difficult to read the measured value in theimage P3 a of the indicator 41.

For example, when a color in the portion of the body composition meter40 corresponding to the area P3 b is white, the entire captured imagebecomes dark, which is likely to cause underexposure of the measuredvalue in the image P3 a of the indicator 41. Alternatively, when a colorin the portion of the body composition meter 40 corresponding to thearea P3 b is black, the entire captured image becomes bright, which islikely to cause overexposure of the measured value in the image P3 a ofthe indicator 41.

On the other hand, when the orientation of the indicator of thehealthcare device is the same as the orientation of a captured image,the position of the image of the indicator the whole of which iscaptured is hardly displaced largely from the center of the capturedimage. FIG. 19 is a diagram of one of examples of an image captured inthe same orientation as that of the indicator. In a picture image P4illustrated in FIG. 19, an image P4 a of the indicator 41 includes theentire indicator 41. In the picture image P4, the image P4 a of theindicator 41 has substantially the same size as that of the image P3 aof the indicator 41 in FIG. 18, and the image P4 a appears in a positionlargely displaced to the left from the center of the picture image P4.However, the right edge of the image P4 a is located at substantiallythe center of the picture image P4. Accordingly, when the mobile phone100 performs the photometric processing based on a central area P4 b asa default operation, exposure adjustment is performed based on thebrightness of an area including the image P4 a of the indicator 41, andtherefore the picture image P4 suitable for reading the measured valuein the image P4 a of the indicator 41 is obtained.

By matching an orientation of a captured image and an orientation of anindicator of a healthcare device included in the image, it is possibleto increase the possibility of obtaining an image suitable for readingthe measured value. For example, when capturing of the indicator of thehealthcare device and analysis of the captured image are repeatedlyperformed until the measured value is successfully read from thehealthcare device, matching the orientation of the image and theorientation of the indicator of the healthcare device enables the systemto shorten the time required to succeed at reading the measured value.

The control for matching an orientation of a captured image and anorientation of an indicator of a healthcare device included in the imagewill be explained with reference to FIG. 20. FIG. 20 is a flowchart ofone of examples of the control for matching the orientation of acaptured image and the orientation of the indicator of a healthcaredevice included in the image. The control illustrated in FIG. 20 isimplemented by the controller 10 executing the healthcare application9D. The control illustrated in FIG. 20 is performed in parallel to theprocessing illustrated in FIG. 9.

As illustrated in FIG. 20, as an initial operation before theorientation of the indicator of a healthcare device is determined, thecontroller 10 performs control based on an assumption that theorientation of the indicator of the healthcare device is horizontal.This is because the orientation of the indicator of the healthcaredevice is in many cases horizontal. The controller 10 may be configuredto perform control based on an assumption that the orientation of theindicator of the healthcare device is vertical, according to the settingor the operation performed by the user.

Specifically, at Step S201, the controller 10 displays an image capturescreen to capture the indicator of the healthcare device in landscapeorientation. Images captured by the camera 13 are continuously displayedon the image capture screen. Subsequently, at Step S202, the controller10 determines whether the measured value of the healthcare device hasbeen detected at Step S104 in the processing illustrated in FIG. 9. Whenthe measured value has not been detected (No at Step S202), thecontroller 10 proceeds to Step S203.

At Step S203, the controller 10 determines the orientation of the mobilephone 100. The orientation of the mobile phone 100 can be determinedusing a detector such as the acceleration sensor 15 and the gyroscope17. When the orientation of the mobile phone 100 is horizontal (Yes atStep S204), the controller 10 returns to Step S202.

When the orientation of the mobile phone 100 is not horizontal (No atStep S204), the controller 10 performs the control so as to change theorientation of the mobile phone 100 to the horizontal orientation.Specifically, at Step S205, the controller 10 displays a messageprompting the user to change the orientation of the mobile phone 100 onthe display 2A. The controller 10 may stop the display of a capturedimage on the image capture screen in addition to the display of themessage. Stopping the display of the image can make the user realizethat the orientation of capturing an image is not appropriate.Thereafter, the controller 10 returns to Step S202.

When the measured value of the healthcare device has been detected atStep S104 in the processing illustrated in FIG. 9 (Yes at Step S202),the controller 10 proceeds to Step S206. At Step S206, the controller 10determines an orientation of the indicator in the image. The orientationdetermined herein is an orientation determined by the long-sidedirection and the short-side direction of the indicator, which has norelation with the orientation in which the measured value is displayed.The orientation of the indicator can be determined based on, forexample, the shape of the area where the measured value is detected inthe processing of FIG. 9, the model of the healthcare device specifiedin the processing of FIG. 9, and the like.

Subsequently, at Step S207, the controller 10 determines whether theorientation of the indicator of the healthcare device matches theorientation of the image being captured. When the orientation of theindicator matches the orientation of the image being captured (Yes atStep S207), the controller 10 proceeds to Step S210.

When the orientation of the indicator of the healthcare device does notmatch the orientation of the image being captured (No at Step S207), thecontroller 10 performs the control so as to match the orientation of thecaptured image to the orientation of the indicator. Specifically, atStep S208, the controller 10 displays a message prompting the user tochange the orientation of the mobile phone 100 on the display 2A, andchanges the orientation of the image capture screen at Step S209. Anexecution order does not matter between Step S208 and Step S209. Thecontroller 10 may stop the display of a captured image on the imagecapture screen in addition to these controls. Thereafter, the controller10 proceeds to Step S210.

At Step S210, the controller 10 determines whether the capturing isterminated. When the capturing is not terminated (No at Step S210), thecontroller 10 returns to Step S206. When the capturing is terminated(Yes at Step S210), the controller 10 ends the processing illustrated inFIG. 20. The controller 10 may always terminate the processing at StepS210.

Embodiment 2

The embodiment 1 has explained the examples in which the mobile phone100 reads the numerical data corresponding to the measured value throughthe character recognition processing from the image obtained bycapturing the measured value displayed in the indicator of thehealthcare device. At this time, any object other than the measuredvalue included in image data (image) captured by the camera affectsadjustment of an exposure amount or the like when an image is acquiredfrom the image data. Therefore, variation may occur in precision ofmeasured value read from the image by the mobile phone 100.Particularly, when the indicator of a healthcare device is formed of a7-segment display that displays a black character on a gray-scale liquidcrystal panel, a measured value in an image acquired from the image databecomes obscure depending on a proportion of a white object included inthe image data. If the precision of the measured value is low, theinformation providing device 300 cannot perform correct analysis on auser's health condition, which leads to degradation of service qualityrelated to healthcare. Therefore, one of examples of the control of themobile phone 100 to precisely read the measured value displayed in theindicator from the image of the captured indicator of the healthcaredevice will be explained below.

The healthcare application 9D provides functions which will be explainedbelow in addition to the functions explained in the embodiment 1. Thehealthcare application 9D provides a function of specifying an area inimage data, in which an image of the indicator of a healthcare deviceappears, based on a luminance distribution of the image data for thehealthcare device captured by the camera 13. The healthcare application9D further provides a function of performing photometric processingtargeted at the specified area.

The controller 10 executes the healthcare application 9D to therebyspecify an area in the image data in which an image of the indicator ofthe healthcare device appears based on the luminance distribution of theimage data for the healthcare device captured by the camera 13.Furthermore, the controller 10 executes the healthcare application 9D tothereby perform the photometric processing targeted at the specifiedarea.

FIG. 21 is a diagram of one of examples of a state in which image datais captured by a camera. FIG. 21 depicts a case where image data for thebody composition meter 40 being a healthcare device is captured. Theuser operates the camera 13 and the mobile phone 100 while checking asituation in which the image data for the indicator 41 of the bodycomposition meter 40 being an object to be captured is displayed on thetouch screen display 2 (display 2A). As illustrated in FIG. 21, thecontroller 10 displays image data I₁ for the body composition meter 40that can be captured by the camera 13 on the display 2A, according to auser's operation.

FIG. 22 is a diagram of one of examples of a luminance distribution ofthe image data illustrated in FIG. 21. A horizontal axis illustrated inFIG. 22 indicates a luminance value and a vertical axis indicates afrequency of the luminance value. The controller 10 calculates theluminance distribution of the image data I₁ illustrated in FIG. 21. Theimage data I₁ illustrated in FIG. 21 includes a luminance value “α” anda luminance value “β” which are peaks, as illustrated in FIG. 22. Theuser operates the camera 13 and the mobile phone 100 by focusing on theindicator 41 of the body composition meter 40 being an object to becaptured. Therefore, the image data I₁ includes a large portion of imageas the indicator 41 of the body composition meter 40. Moreover, when theindicator 41 of the body composition meter 40 is formed of a liquidcrystal panel that displays character information with a gray scale,luminance values of pixels forming the indicator 41 are basically thesame values as each other. Thus, it is possible to determine that eitherone of the luminance value “α” and the luminance value “β” which arepeaks is quite likely to be the indicator 41.

FIG. 23 is a diagram of one of examples of positions of pixels, in theimage data I₁, having a peak luminance value in the luminancedistribution illustrated in FIG. 22. FIG. 23 depicts one of examples ofpositions of pixels P_(α), in the image data I₁, having the luminancevalue “α” which is one of peaks in the luminance distributionillustrated in FIG. 22.

FIG. 24 is a diagram of one of examples of an area to be cut out as acandidate of an area in which the image of the indicator 41 of the bodycomposition meter 40 appears. Because the user operates the camera 13and the mobile phone 100 by focusing on the indicator 41 of the bodycomposition meter 40 being an object to be captured, the image data I₁can be predicted to include the large portion of image as the indicator41 of the body composition meter 40. Therefore, the controller 10 refersto the positions (FIG. 23) of the pixels P_(α), in the image data I₁,having the luminance value “α” which is one of peaks in the luminancedistribution, and cuts out an area A_(α) having a concentration of thepixels P_(α) illustrated in FIG. 24 as a candidate of the area in whichthe image of the indicator 41 of the body composition meter 40 appears,from the image data I₁. For example, when the candidate of the area inwhich the image of the indicator 41 of the body composition meter 40appears is to be cut out from the image data I₁, the controller 10 cutsout, for example, a minimum rectangle including an area having aconcentration of the pixels P_(α) which have one of the peak luminancevalues. The controller 10 can cut out an area in any one of variousshapes such as a circle, an ellipse, and a triangle according to, forexample, the shape of the area having a concentration of the pixelsP_(α).

The controller 10 performs the character recognition processing on thearea A_(α) cut out from the image data I₁ as a candidate of the area inwhich the image of the indicator 41 of the body composition meter 40appears. When the character information can be recognized in the areaA_(α), the controller 10 specifies the area A_(α) as an area in whichthe image of the indicator 41 of the body composition meter 40 appears.

When the area A_(α) can be specified as an area in which the image ofthe indicator 41 of the body composition meter 40 appears, thecontroller 10 performs the photometric processing targeted at the areaA_(α). When the photometric processing targeted at the area A_(α) is tobe performed, the controller 10 may use any one of division photometry,centrally weighted photometry, and spot photometry. When the photometricprocessing is performed using the division photometry, the controller 10divides the area A_(α) into a plurality of areas, and calculates acontrol value used to determine an exposure amount from an average oflight amounts in the divided areas. When the photometric processing isperformed using the centrally weighted photometry, the controller 10weights the light amount in a central area of the area A_(α) tocalculate a control value used to determine an exposure amount. When thephotometric processing is performed using the spot photometry, thecontroller 10 calculates a control value used to determine an exposureamount based on a light amount in a slightly small portion of the areaA_(α).

FIG. 25 is a diagram for explaining one of examples of a processingprocedure in which the mobile phone 100 specifies an area, in which theimage of the indicator of the healthcare device appears, from the imagedata captured by the camera 13 and performs photometric processingtargeted at the specified area. The processing procedure illustrated inFIG. 25 is started by the controller 10 that activates the healthcareapplication 9D. The activation of the healthcare application 9D isperformed in response to, for example, detection of a user operationsimilarly to the embodiment 1.

The controller 10 activates the camera 13 after the activation of thehealthcare application 9D, and calculates a luminance distribution (seeFIG. 22) of the image data I₁ (see FIG. 21, etc.) being captured by thecamera 13 (Step S301). Subsequently, the controller 10 selects one (forexample, α) of peak luminance values in the luminance distribution (StepS302).

Subsequently, the controller 10 cuts out an area (for example, A_(α),see FIG. 24), in the image data I₁, having a concentration of the pixels(for example, P_(α)) which have the peak luminance value (Step S303).

The controller 10 then performs the character recognition processing onthe area (for example, A_(α)) cut out at Step S303 and determineswhether the character information can be detected in the area (StepS304).

When it is determined that the character information can be detected inthe area cut out at Step S303 (Yes at Step S304), the controller 10specifies the area cut out at Step S303 as an area, in the image dataI₁, in which the image of the indicator 41 of the body composition meter40 appears (Step S305).

Subsequently, the controller 10 performs the photometric processingtargeted at the area specified at Step S305 (Step S306). Then, thecontroller 10 adjusts the exposure amount of when the image data I₁ iscaptured and the image is acquired, based on the result of thephotometric processing (Step S307), and ends the processing procedureillustrated in FIG. 25.

At Step S304, when it is determined that the character informationcannot be detected in the area cut out at Step S303 (No at Step S304),the controller 10 determines whether there is any unselected one in thepeak luminance values (Step S308). When it is determined that there isan unselected one (Yes at Step S308), the controller 10 returns to StepS302, selects one luminance value from unselected luminance values, andperforms the above-described procedure. Meanwhile, when it is determinedthat there is no unselected one (No at Step S308), then the controller10 ends the processing procedure illustrated in FIG. 25 without anychange.

As explained above, in the embodiment 2, the mobile phone 100 executesthe healthcare application 9D to thereby specify an area in the imagedata I₁ in which the image of the indicator of the healthcare deviceappears, based on the luminance distribution of the image data I₁ of thebody composition meter 40 captured by the camera 13. Furthermore, thecontroller 10 executes the healthcare application 9D to thereby performthe photometric processing targeted at the specified area. Therefore,according to the embodiment 2, the measured value displayed in theindicator can be precisely read from the image obtained by capturing theindicator of the healthcare device.

In the embodiment 2, the mobile phone 100 cuts out an area in the imagedata I₁ corresponding to the peak luminance value in the luminancedistribution as a candidate of an area in which the image of theindicator 41 appears. When the character information (for example,7-segment) can be detected in the area cut out as a candidate, themobile phone 100 specifies the area in the image data I₁ correspondingto the peak luminance value as an area in which the image of theindicator appears. The user operates the camera 13 and the mobile phone100 by focusing on the indicator 41 of the body composition meter 40being an object to be captured, and therefore the image data I₁ can bepredicted to include a large portion of image as the indicator 41 of thebody composition meter 40. Thus, a manner of performing the processingwhile determining the relevant area in the image data I₁ correspondingto the peak luminance value in the luminance distribution as an area inwhich the indicator 41 appears is effective. However, if any characterinformation can be recognized in the relevant area, an area in which theimage of the indicator 41 appears can be more surely specified.

In the embodiment 2, the area of the indicator 41 is specified based onthe luminance distribution of the image data I₁ of the body compositionmeter 40. Therefore, for example, even when the position of the camera13 does not face the body composition meter 40 directly, the measuredvalue displayed in the indicator 41 can be precisely read. In otherwords, even when the image of the indicator 41 is captured at an angle,the photometric processing targeted at the indicator 41 is performed,and thus the measured value displayed in the indicator can be readwithout any influence such as the light reflected from the indicator 41or the light reflected in the indicator 41.

The embodiment 2 has explained the examples of specifying the area inthe image data I₁ in which the image of the indicator 41 of the bodycomposition meter 40 appears based on the luminance distribution of theimage data I₁ of the healthcare device 40 being captured by the camera13. For example, the body composition meter 40 may be provided with apattern design so as to specify an area in the image data I₁ in whichthe image of the indicator 41 of the body composition meter 40 appears.

FIG. 26 is a diagram of one of examples of a pattern design forspecifying an area in the image data in which the image of the indicator41 of the body composition meter 40 appears. As illustrated in FIG. 26,the pattern design is formed of three rectangles 41 a arranged to beinscribed in three corners out of the four corners of the indicator 41of the body composition meter 40, and a line 41 b surrounding theindicator 41. When the image data for the body composition meter 40 iscaptured by the camera 13, the controller 10 recognizes the patterndesign illustrated in FIG. 26, to thereby specify an area in the imagedata in which the image of the indicator 41 of the body compositionmeter 40 appears. The controller 10 may read the pattern designillustrated in FIG. 26 through pattern matching, or may read the patterndesign illustrated in FIG. 26 by extracting edges from the image data.

An area of the indicator 41 may be specified not only by specifying thearea of the indicator 41 in which the image of the indicator 41 of thebody composition meter 40 appears by using the pattern designillustrated in FIG. 26 but also by detecting a display content of theindicator 41 and/or a change of the display content. For example, thecontroller 10 performs the character recognition processing on the imagedata for the body composition meter 40 being captured by the camera 13,and, when characters such as “in measuring” can be detected, specifiesan area in which the image of the indicator 41 appears by using theposition of the characters as a target. Alternatively, the controller 10calculates an inter-frame difference of the image data for the bodycomposition meter 40 being captured by the camera 13, and, when thechange of the display content displayed in the indicator 41 duringexecution of measurement can be detected, specifies the area in whichthe image of the indicator 41 appears by using the changed portion as atarget. The change of the display content includes a change of a numeraluntil a measured value of body weight is fixed, a change from a displayof body weight to a display of body fat, and the like.

Alternatively, when the image data for the indicator 41 of the bodycomposition meter 40 is captured by the camera 13, a guide frame or thelike used to be fitted to the indicator 41 may be displayed on thedisplay 2A. In this case, an area inside the guide frame, of the imagedata captured by the camera 13, is specified as an area in which theimage of the indicator 41 appears.

The embodiment 2 has explained the examples of specifying the area ofthe indicator 41 of the body composition meter 40; however, the same wayas that of the body composition meter 40 can be used to specify anindicator of healthcare devices other than the body composition meter40, such as a thermometer, a blood pressure monitor for measuring bloodpressure, a pulse meter for measuring pulses, a pedometer for countingsteps, and an activity meter for measuring an amount of activity due toexercise including walking.

In the embodiment 2, when a color temperature of the indicator 41 of thebody composition meter 40 or a type of the light source is previouslyfound out, the mobile phone 100 can also adjust white balance accordingto the color temperature in the area of the indicator 41 or the type ofthe light source.

In the embodiment 2, the mobile phone 100 may change ISO sensitivityaccording to the result of photometric processing targeted at the areain which the image of the indicator 41 appears.

In the embodiment 2, it may be configured such that the mobile phone 100narrows down an area, in which the indicator 41 in the image data forthe body composition meter 40 being captured by the camera 13 appears,to some extent by using the used device information 9X, thedetermination information 9Y, and the like, and then specifies an areain which the indicator 41 appears, based on a luminance distribution ofthe narrowed down area.

The mobile phone 100 may perform the same processing as that of theembodiment 1 (Yes at Step S105 to Step S122) using the used deviceinformation 9X, the determination information 9, and the like after themeasured value is acquired from the image captured by controlling theexposure amount based on the result of photometric processing targetedat the area of the indicator 41.

Embodiment 3

The mobile phone 100 may perform the control so that there is lowpossibility that the shadow of the mobile phone 100 or the mobile phone100 itself unexpectedly appears in an image obtained by capturing theindicator of the healthcare device. How the shadow of the mobile phone100 or the mobile phone 100 itself unexpectedly appears in the imagewill be explained below with reference to FIG. 27 to FIG. 30.

FIG. 27 is a diagram of one of examples of how the mobile phone 100itself unexpectedly appears in an image. FIG. 28 is a diagram of one ofexamples of an image in which the mobile phone 100 itself unexpectedlyappears. In the example illustrated in FIG. 27, the mobile phone 100 isheld by the user substantially right above the body composition meter 40in such a manner as to face the body composition meter 40 directly inorder to capture an approximately rectangular indicator 41 (including asquare) of the body composition meter 40 using the camera 13. In thiscase, when the surface of the indicator 41 is covered with a materialthat reflects light such as glass or transparent resin, the surface ofthe indicator 41 functions as a kind of mirror, and this may cause themobile phone 100 itself to unexpectedly appear in an image captured bythe camera 13.

For example, in a picture image P5 illustrated in FIG. 28, an image P5 bof the mobile phone 100 unexpectedly appears inside an image P5 a of theindicator 41. Such an unexpected appearance as above partially changesthe luminance or the like of the image P5 a, which may make it difficultto read the measured value included in the image P5 a. Particularly,when a reflective display panel without a backlight is used for theindicator of a healthcare device, an amount of change in luminance ofthe indicator is originally small, and therefore the measured value maybecome difficult to be read due to a luminance difference produced bythe unexpected appearance.

FIG. 29 is a diagram of one of examples of how the shadow of the mobilephone 100 unexpectedly appears in an image. FIG. 30 is a diagram of oneof examples of an image in which the shadow of the mobile phone 100unexpectedly appears. In the example illustrated in FIG. 29, the lightsource L1 is provided on the ceiling and the body composition meter 40is placed on the floor. The mobile phone 100 is held by the user inbetween the light source L1 and the body composition meter 40 in orderto capture the indicator 41 of the body composition meter 40 using thecamera 13. In this case, a shadow of the mobile phone 100 occurs overthe body composition meter 40 due to the light from the light source L1,and the shadow is likely to unexpectedly appear in the image captured bythe camera 13. For the sake of simplicity, a user's body (especially,hand) by which the mobile phone 100 is held is not considered.

For example, in a picture image P6 illustrated in FIG. 30, an image P6 bof the mobile phone 100 unexpectedly appears therein to overlap part ofan image P6 a of the indicator 41. The unexpected appearance of theshadow causes the luminance or the like of the image P6 a to partiallychange, which may make it difficult to read the measured value includedin the image P6 a.

To prevent the unexpected appearance of the shadow of the mobile phone100 or of the mobile phone 100 itself in the image, it is consideredthat the mobile phone 100 is inclined at a position away from rightabove the indicator 41 and the indicator 41 is captured, as illustratedin FIG. 31. FIG. 31 is a diagram of one of examples of a state in whichthe unexpected appearance does not occur. FIG. 32 is a diagram of one ofexamples of an image captured in the state illustrated in FIG. 31. In apicture image P7 illustrated in FIG. 32, neither the shadow of themobile phone 100 nor the mobile phone 100 itself unexpectedly appears inan image P7 a of the indicator 41. Because it is captured in theinclined state, the image P7 a is distorted to an isosceles trapezoid inwhich its upper base is shorter than its bottom base, but by performingtrapezoidal correction processing on the image P7 a, the image P7 a canbe corrected to a rectangle suitable for reading the measured value.

To cause the user to perform image capture in the state illustrated inFIG. 31, the control is simply performed so that the image of asubstantially rectangular indicator of the healthcare device in an imagecaptured by the camera 13 is formed to a trapezoid due to a differencein perspective between its upper side and lower side, as the pictureimage P7 illustrated in FIG. 32. Therefore, the mobile phone 100 candisplay an image capture screen S1 as illustrated in FIG. 33 on thedisplay 2A when the indicator of the healthcare device is captured. FIG.33 is a diagram of one of examples of the image capture screen S1.

The image capture screen S1 has an image display area S1 a that occupiesan upper half thereof and a message display area S1 b that occupies alower half thereof. The image display area S1 a is an area in which animage captured by the camera 13 is continuously displayed. The imagedisplay area S1 a includes a frame w1. The frame w1 has a shape of anisosceles trapezoid in which the upper base is shorter than the bottombase similarly to the image P7 a of the indicator 41 in the pictureimage P7. The message display area S1 b is an area in which a messagefor the user is displayed. In the example illustrated in FIG. 33, amessage prompting the user to capture an image so that the indicator 41of the body composition meter 40 just fits in the frame is displayed inthe message display area S1 b. The message display area S1 b may furtherinclude a message prompting the user to incline the mobile phone 100.

By displaying the image capture screen S1 as described above on thedisplay 2A, the frame w1 functions as a frame to which an outer shape ofthe image of the indicator of the healthcare device in the imagecaptured by the camera 13 should be fitted. In other words, the useradjusts the position and inclination of the mobile phone 100 so that theouter shape of the image of the indicator of the healthcare device fitsthe frame w1 while viewing the image capture screen S1. Consequently,the capturing is performed in the state as illustrated in FIG. 31.

FIG. 34 is a diagram of one of examples of an image capture screen in astate of facing the healthcare device substantially directly. FIG. 35 isa diagram of one of examples of the image capture screen in the stateillustrated in FIG. 31. For example, when the image capture is performedin the state in which the mobile phone 100 directly faces the bodycomposition meter 40 substantially right above the body compositionmeter 40 as illustrated in FIG. 27 or FIG. 29, the image of theindicator 41 does not fit the frame w1 as illustrated in the example ofFIG. 34. Therefore, the user can easily understand that the position andthe inclination of the mobile phone 100 are not appropriate. The userthen adjusts the position and the inclination of the mobile phone 100 sothat the outer shape of the image of the indicator 41 fits the frame w1as illustrated in the example of FIG. 35, thus achieving the state inwhich the picture image P7 illustrated in FIG. 32 is obtained, that is,achieving the state illustrated in FIG. 31.

Displaying the image capture screen S1 including the trapezoidal framew1 in the above manner allows the user to perform capturing in asuitable condition without giving a complicated instruction to the user.Moreover, by using this technique, it can be expected that the shape ofthe image of the indicator 41 included in the captured image becomessubstantially constant. Therefore, a correction coefficient oftrapezoidal correction for correcting the shape of the image of theindicator 41 to a rectangle in order to read the measured value includedin the image can be determined in advance, thus reducing the load ofimage processing.

A control for reducing a possibility that the unexpected appearance ofthe shadow of the mobile phone 100 or the mobile phone 100 itself occurswill be explained with reference to FIG. 36. FIG. 36 is a flowchart ofone of examples of control for reducing a possibility for occurrence ofthe unexpected appearance. The control illustrated in FIG. 36 isimplemented by the controller 10 executing the healthcare application9D.

As illustrated in FIG. 36, at Step S401, the controller 10 displays theimage capture screen S1 including the trapezoidal frame w1 on thedisplay 2A. Then, at Step S402, the controller 10 sets a correctioncoefficient to perform trapezoidal correction on the captured image. Thecorrection coefficient set herein is, for example, a coefficient forcorrecting the shape of the frame w1 to a rectangle. Thereafter, at StepS403, the controller 10 performs the processing illustrated in FIG. 9.In the processing, the controller 10 performs trapezoidal correction onthe image captured by the camera 13 using the correction coefficient setat Step S402 before reading the numeral, symbol, and the like includedin the image captured by the camera 13.

The controller 10 may dynamically change the shape of the frame w1 to bedisplayed on the display 2A during execution of the processingillustrated in FIG. 9. For example, when the distortion of the image ofthe indicator 41 in the image is too large to read the numerals or thelike, the controller 10 may perform at least one of the control forelongating the upper base of the frame w1 and the control for widening adistance between the upper base and the lower base of the frame w1. Bychanging the shape of the frame w1 in this manner, it is possible tolead the user to change the position and the inclination of the mobilephone 100 in a direction in which the distortion of the frame w1 isreduced, in other words, to change them so that the mobile phone 100 andthe body composition meter 40 face each other more directly. Forexample, when the numeral and the like included in the image of theindicator 41 cannot be read entirely, it is possible to determine thatthe distortion of the image is too large.

Alternatively, when the numeral and the like are difficult to be readdue to the unexpected appearance of the shadow of the mobile phone 100or of the mobile phone 100 itself in the image, the controller 10 mayperform at least one of the control for shortening the upper base of theframe w1 and the control for narrowing the distance between the upperbase and the lower base of the frame w1. By changing the shape of theframe w1 in this manner, it is possible to lead the user to change theposition and the inclination of the mobile phone 100 in a direction inwhich the unexpected appearance is difficult to occur, in other words,to change them so that the mobile phone 100 and the body compositionmeter 40 face each other less directly. For example, when the numeraland the like included in the image of the indicator 41 cannot be readpartially, it is possible to determine that the unexpected appearanceoccurs.

When the shape of the frame w1 is dynamically changed in this manner,the controller 10 changes the correction coefficient set at Step S402 inaccordance with the shape of the frame w1 after the change.

In the above explanation, the example of displaying the trapezoidalframe w1 is described as one of examples of display control for reducingthe possibility that the shadow of the mobile phone 100 or the mobilephone 100 itself unexpectedly appears in the image; however, the displaycontrol is not limited thereto. For example, the controller 10 maydisplay the image, as the image capture screen, obtained by performingthe trapezoidal correction thereon using the correction coefficient tobe set at Step S402 illustrated in FIG. 36 on the display 2A instead ofdisplaying the trapezoidal frame w1 thereon. With this control, forexample, the picture image P7 (see FIG. 32) captured in the stateillustrated in FIG. 31 is displayed as an image with no distortion likea picture image P8 illustrated in FIG. 37. FIG. 37 is a diagram of oneof examples in which trapezoidal correction is performed on the imagecaptured in the state illustrated in FIG. 31. An image P8 a of theindicator 41 illustrated in FIG. 37 is an image in which the shape ofthe image P7 a distorted to the isosceles trapezoid with the upper baseshorter than the lower base is corrected to the rectangle through thetrapezoidal correction.

In this case, the mobile phone 100 displays an image capture screen S2as illustrated in FIG. 38 on the display 2A when the indicator of thehealthcare device is captured. FIG. 38 is a diagram of another exampleof the image capture screen. The image capture screen S2 has an imagedisplay area S2 a that occupies an upper half thereof and a messagedisplay area S2 b that occupies a lower half thereof. The image displayarea S2 a is an area in which an image captured by the camera 13 iscontinuously displayed. The image display area S2 a includes a frame w2.The frame w2 has a rectangular shape. The image display area S2 a doesnot have to include the frame w2. The message display area S2 b is anarea in which a message for the user is displayed. In the exampleillustrated in FIG. 38, a message prompting the user to capture an imageso that the indicator 41 of the body composition meter 40 just fits inthe frame is displayed in the message display area S2 b.

FIG. 39 is a diagram of one of examples of an image capture screen in astate of facing the healthcare device substantially directly. FIG. 40 isa diagram of one of examples of the image capture screen in the stateillustrated in FIG. 31. For example, when the capturing is performed inthe state in which the mobile phone 100 directly faces the bodycomposition meter 40 substantially right above the body compositionmeter 40 as illustrated in FIG. 27 or FIG. 29, the image of theindicator 41 is distorted to a trapezoidal shape, as illustrated in theexample of FIG. 39, in which the upper side is longer than the lowerside because the trapezoidal correction is performed thereon, andfurthermore the image does not fit the frame w2. Therefore, the user caneasily understand that the position and the inclination of the mobilephone 100 are not appropriate. The user then adjusts the position andthe inclination of the mobile phone 100 so that the outer shape of theimage of the indicator 41 fits to the frame w2 while eliminating thedistortion of the image as illustrated in the example of FIG. 40, thusachieving the state in which the picture image P8 illustrated in FIG. 37is obtained, that is, achieving the state illustrated in FIG. 31.

Displaying the image after the trapezoidal correction in the abovemanner also allows the user to perform capturing in the suitablecondition without giving a complicated instruction to the user.Moreover, by using this technique, the user adjusts the position andinclination of the mobile phone 100 so as to reduce the distortion inthe displayed image, that is, in the image from which the measured valueis read, and this enables accurate reading of the measured value usingthe image with less distortion.

In other words, the present embodiment is implemented by using thepsychology of the user such as “I want the mobile phone to face the bodycomposition meter 40 directly”. In other words, it is implemented byusing the psychology of the user such that when the image of theindicator 41 is distorted as illustrated in FIG. 39, this causes theuser to make a mistake that the mobile phone 100 does not face the bodycomposition meter 40 directly and he/she tries inclining the mobilephone 100 as illustrated in FIG. 31.

The embodiment represents the examples of displaying the trapezoidalframe w1 and the image of the indicator 41 illustrated in FIG. 39 in atrapezoid in which the upper side and the lower side are parallel toeach other. However, the embodiment is not limited thereto if it ispossible to make the user be aware that the mobile phone 100 and thebody composition meter 40 should not face each other directly.

The embodiments have explained the mobile phone as one of examples ofthe electronic device; however, the electronic device according to theappended claims is not limited to the mobile phone. The electronicdevice according to the appended claims may be any mobile electronicdevice other than the mobile phone. Examples of the mobile electronicdevice includes, but are not limited to, a tablet, a mobile personalcomputer, a digital camera, a media player, an electronic book reader, anavigator, and a gaming device. The electronic device according to theappended claims may be any electronic device other than the mobileelectronic device.

Although the art of appended claims has been described with respect tospecific embodiments for a complete and clear disclosure, the appendedclaims are not to be thus limited but are to be construed as embodyingall modifications and alternative constructions that may occur to oneskilled in the art which fairly fall within the basic teaching hereinset forth.

1. An electronic device, comprising: a camera; a controller configuredto determine whether a measured value of a healthcare device is includedin an image captured by the camera; and a communication unit configuredto transmit information related to the measured value to an informationproviding device when the measured value is included in the image. 2.The electronic device according to claim 1, wherein the controllerperforms control so as to match an orientation of the image captured bythe camera to an orientation of an indicator indicating the measuredvalue of the healthcare device.
 3. The electronic device according toclaim 2, wherein the controller performs the control when it isdetermined that the measured value of the healthcare device is includedin a video being captured by the camera.
 4. The electronic deviceaccording to claim 2, further comprising a display for displaying avideo being captured by the camera, wherein, when an orientation of thevideo being captured by the camera does not match the orientation of theindicator, the controller changes the orientation of the video.
 5. Theelectronic device according to claim 2, further comprising a display fordisplaying a message prompting to change the orientation of theelectronic device when the orientation of the video being captured bythe camera does not match the orientation of the indicator.
 6. Theelectronic device according to claim 4, wherein, when the orientation ofthe video being captured does not match the orientation of theindicator, the controller causes the display not to display the videobeing captured.
 7. (canceled)
 8. The electronic device according toclaim 1, further comprising a display for displaying a video beingcaptured by the camera, wherein the controller controls displaying bythe display so that an image of the indicator indicating the measuredvalue of the healthcare device in the video being captured by the camerais formed to a trapezoid.
 9. The electronic device according to claim 8,wherein the controller cause the display to display a trapezoidal frame,as a frame used to fit an outer shape of the image of the indicator inthe video being captured by the camera.
 10. The electronic deviceaccording to claim 8, wherein the controller is configured to read themeasured value indicated in the indicator from an image in which a shapeof the image of the indicator captured by the camera is corrected, anddisplay the corrected image on the display.
 11. The electronic deviceaccording to claim 1, further comprising a display configured to, whenthe camera faces the indicator of the healthcare device directly,display the indicator being captured by the camera in such a manner asto be formed to a trapezoid.
 12. An information providing systemcomprising an electronic device and an information providing device,wherein the electronic device includes: a camera; a first controllerconfigured to determine whether a measured value of a healthcare deviceis included in an image captured by the camera; and a communication unitconfigured to transmit information related to the measured value to theinformation providing device when the measured value is included in theimage, and the information providing device includes: a storageconfigured to store the information related to the measured value; and asecond controller configured to perform analysis based on theinformation related to the measured value.
 13. The information providingsystem according to claim 12, wherein the first controller performscontrol so as to match an orientation of the image captured by thecamera to an orientation of an indicator indicating the measured valueof the healthcare device.
 14. (canceled)
 15. The information providingsystem according to claim 12, wherein the electronic device furtherincludes a display for displaying a video being captured by the camera,and the first controller controls displaying by the display so that animage of the indicator indicating the measured value of the healthcaredevice in the video being captured by the camera is formed to atrapezoid.
 16. The information providing system according to claim 12,wherein the electronic device further includes a display configured to,when the camera faces the indicator of the healthcare device directly,display the indicator being captured by the camera in such a manner asto be formed to a trapezoid.
 17. A control method of an electronicdevice with a camera, the control method comprising: determining whethera measured value of a healthcare device is included in an image capturedby the camera; and performing analysis for providing information basedon information related to the measured value.
 18. The control methodaccording to claim 17, further comprising: performing control so as tomatch an orientation of the image captured by the camera to anorientation of an indicator indicating the measured value of thehealthcare device.
 19. The control method according to claim 18, whereinthe performing control is executed when it is determined that themeasured value of the healthcare device is included in a video beingcaptured by the camera.
 20. (canceled)
 21. The control method accordingto claim 17, further comprising: displaying a video being captured bythe camera on a display provided in the electronic device; andcontrolling displaying by the display so that an image of the indicatorindicating the measured value of the healthcare device in the videobeing captured by the camera is formed to a trapezoid.
 22. The controlmethod according to claim 17, further comprising: displaying, when theelectronic device faces the indicator of the healthcare device directly,the indicator being captured by the camera in such a manner as to beformed to a trapezoid on a display provided in the electronic device.23. A non-transitory storage medium that stores a control program thatcauses, when executed by an electronic device with a camera, theelectronic device to execute: determining whether a measured value of ahealthcare device is included in an image captured by the camera; andperforming analysis for providing information based on informationrelated to the measured value. 24-28. (canceled)